Skip to content

Comments

Single affinity non async zc#8

Merged
ming1 merged 4 commits intomainfrom
single-affinity-non-async-zc
Aug 18, 2025
Merged

Single affinity non async zc#8
ming1 merged 4 commits intomainfrom
single-affinity-non-async-zc

Conversation

@ming1
Copy link
Collaborator

@ming1 ming1 commented Aug 18, 2025

No description provided.

ming1 added 4 commits August 18, 2025 09:19
Prepare for supporting single cpu affinity and non-asyn/await zero copy
support.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Changes Made:

1. Updated Cargo.toml: Changed libublk dependency to use the main branch from
GitHub instead of crates.io to get the latest version with UBLK_DEV_F_SINGLE_CPU_AFFINITY support.

2. Added --multi-cpus-affinity flag: Added a new command-line option to GenAddArgs
in src/args.rs:69 that allows users to opt-out of single CPU affinity.

3. Applied single CPU affinity by default: Modified the new_ublk_ctrl method in
src/args.rs:206-211 to automatically apply UBLK_DEV_F_SINGLE_CPU_AFFINITY unless
the --multi-cpus-affinity flag is specified.

How it works:
- Default behavior: All ublk devices now use single CPU affinity for improved performance
- Opt-out option: Users can specify --multi-cpus-affinity to use the previous multi-CPU behavior
- Universal application: The feature applies to all target types (loop, null, qcow2, compress, zoned)

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
1. Updated args.rs: Modified zero-copy validation to allow null targets alongside loop targets

2. Enhanced null.rs with zero-copy support:
  - Added handle_queue_tag_async_null_zc() function that uses submit_io_cmd_with_auto_buf_reg()
    with UBLK_AUTO_BUF_REG_FALLBACK flag
  - Added handle_queue_tag_async_null() function for regular async operations
  - Modified q_async_fn() to detect auto buffer registration support and select appropriate handler
  - Added validation requiring --async-await when zero-copy is enabled for null targets

Also add sync zc:
  1. Added q_sync_zc_fn() function:
    - Creates auto buffer registration list for all queue depths
    - Uses UblkQueue::submit_fetch_commands_with_auto_buf_reg() for initialization
    - Uses UblkQueue::complete_io_cmd_with_auto_buf_reg() in the I/O handler
    - Handles the shared data using Rc to satisfy borrow checker requirements
  2. Updated queue handler logic:
    - Added condition to use q_sync_zc_fn() when UBLK_F_AUTO_BUF_REG flag is set and async mode is not enabled
    - Maintains existing behavior for async and regular sync modes
  3. Removed async-await restriction:
    - Zero-copy can now work in both synchronous and asynchronous modes
    - Aligns with the new synchronous zero-copy implementation

Key Features:
- Zero-copy support: Null target now supports --zero-copy flag using UBLK_F_AUTO_BUF_REG
- Automatic buffer registration: Uses ublk_auto_buf_reg with fallback mode
- Validation: Requires --async-await flag when zero-copy is enabled
- Performance optimization: Eliminates buffer copies between kernel and userspace

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Changes Made to src/loop.rs

1. Added lo_handle_io_cmd_sync_zc() function (src/loop.rs:229-253):
  - Zero-copy version of the synchronous I/O command handler
  - Uses UblkQueue::complete_io_cmd_with_auto_buf_reg() instead of complete_io_cmd()
  - Uses __lo_make_io_sqe_zc() for zero-copy SQE generation instead of __lo_make_io_sqe()

2. Added q_zc_fn() function (src/loop.rs:281-301):
  - Synchronous zero-copy queue function using auto buffer registration
  - Creates auto buffer registration list for all queue depths
  - Uses UblkQueue::submit_fetch_commands_with_auto_buf_reg() for initialization
  - Uses shared Rc pattern to handle borrow checker requirements

3. Updated queue handler logic (src/loop.rs:435-447):
  - Added condition to use q_zc_fn() when UBLK_F_AUTO_BUF_REG flag is set and async mode is not enabled
  - Maintains existing behavior for async (q_a_fn()) and regular sync (q_fn()) modes

4. Removed async-await restriction (src/loop.rs:431-433):
  - Zero-copy can now work in both synchronous and asynchronous modes
  - Loop target now supports --zero-copy flag without requiring --async-await

The implementation follows the same patterns as both the existing loop async zero-copy
code and the null target's synchronous zero-copy implementation, ensuring consistency
across the codebase. The loop target now supports zero-copy mode with --zero-copy flag
in both synchronous and asynchronous modes, providing better performance through
automatic buffer registration.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
@ming1 ming1 merged commit d37f8af into main Aug 18, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant